Note: The following concludes an eight-part serialization of selected content from Steve Morlidge's The Little (Illustrated) Book of Operational Forecasting.

Good forecasts don’t always ‘look right’

Many forecasters believe that they can tell how good a forecast is by ‘eyeballing’ it. Good forecasts just ‘look right’ or so they would like to believe, which justifies them manually overriding the system generated forecast or playing around with its parameters until they get the answer they want.

In my experience ‘looking right usually this means that the forecast must look somewhat like the past. But this assumption is completely false.

The reason for this is that the past will contain noise but a good forecast will only be made up of a signal, and so may well look completely different. So, good forecasts will be those that DO NOT look like the past. They will look unnatural because they are unnatural – they don’t have the ‘rough’ look that we recognise in nature. Good forecasts do usually not look ‘wiggly’.

This is most clearly illustrated when we have a stable signal which is accompanied by a lot of noise.

I know of software vendors who have lost sales because prospective customers don’t believe that a straight line is the best forecast possible. ‘I’m not paying you for that!’, they said. Also, forecasters often tamper with system generated forecast that they ‘don’t like the look of’ by making manual adjustments of playing around with system parameters.

There is one exception to the rule that ‘good forecasts don’t have to look right’. It is sometimes helpful to look at the results of low level forecasts in aggregate. At a high level noise is damped down so visual inspection is a more reliable guide. If low level forecasts in aggregate show trends that are inconsistent with the past a demand manager should satisfy herself that there is good reason for this.

TAKEOUT

Don’t judge forecasts on the basis of whether they ‘look like the past’. Good forecasts will probably look very different since they discount the noise in the historic record.